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Abstract: Mobile learning has moved in the last decade from being a small, scattered research 
interest to being viewed by many international agencies as a way of delivering their humanitarian 
missions to the developing contexts of the global South. 

This paper explores and documents fundamental concepts and concerns that characterize or 
perhaps jeopardise the relationships between the 'old' research communities and 'new' policy 
maker communities working to improve the nature and scope of learning in the developing 
contexts of the global South using personal mobile digital technologies. As becomes apparent, 
these concepts and concerns are relevant and interesting across a broader range of domains, 
touching perhaps under-privilege and access to education and technology in both the global North 
and the global South, the uses of technology to extend, enhance and transform learning and the 
various pressures and determinants of policy-making and of the public funding of research. 
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Introduction 

This piece grows out of personal experience and reflection and is consequently partial and 
subjective; it is however an area that is difficult for researchers to explore objectively, 
analytically and comprehensively, for two main reasons. The first is that the area is necessarily 
subjective for researchers because researchers as a community themselves are part of it not 
separate from it, they are players and stakeholders in the relations and transactions being 
explored and discussed. The second is that the methodological tools do not really exist to 
explore and analyse these relations naturalistically; these relations and the transactions that 
comprise them are verbal, fluid, informal, privileged and political. Policy-makers are 
improbable research subjects. 

This piece is an attempt to explain and explore what seems to be going on as these two 
communities and those alongside them try to build better and more productive relationships. 

It is partly exposition and explanation, partly exhortation and partly critique. 

As some context for the development of this piece, I should mention briefly the process that 
catalysed it. Annually UNESCO in Paris holds a Mobile Learning Week. Leading up to the 
Week in 2015, UNESCO proposed a symposium that explored the interface between research 
and policy, assuming that both parties understood the other only poorly. This was a 
worthwhile exercise in itself but also a source of research data that might lead to findings that 
were useful to the two communities. 
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This document draws on two sets of qualitative data. In both cases, these are drawn from small 
self-selected samples, namely people who attended the symposium, more than 80 of them, and 
who chose to speak, and people who subsequently offered responses to an online survey. 

These people were a cross-section of experienced researchers, programme managers and 
policy makers, from both the global North and the global South. There was a sequence and 
development in that responses from the symposium informed the questions subsequently 
posed in the survey, seeking to probe for examples, counter examples and opinions. The aim 
was not to produce outcomes or findings that were objective or representative but to illustrate 
the breadth of experience, ideas and practices. This aim underpinned how the data as gathered 
and how it was analysed, and how subsequently presented. No formal research method or 
theory was used or assumed. 

The moderators for each segment of the symposium were provided with scripts, in effect a 
focus group protocol, and used it to even-handedly move through their respective panels, 
seeking responses in turn and then throwing the topic open to the other participants. The vast 
majority of all the segments was audio recorded and transcribed, and notes taken 
independently verbatim. 

Participants were emailed to participate in an online survey that consisted of a series of open- 
ended questions directly related to one or other of the themes from the symposium. There 
were twenty-eight respondents and most made substantial free-text answers to most of the 
twenty questions. We must recognise that the vast majority of these respondents identified 
themselves as researchers. Their responses must be seen in this context but are valuable 
nevertheless and in most instances draw on experience in the field and on experience of 
engaging with policy makers. These represent the deeply flawed attempts to use recognised 
methods to capture qualitative data objectively. They were however also informed by 
comparable contacts and collaborations with DFID, USAID, World Bank and World Economic 
Forum. 

What is Mobile Learning? 

We must start by introducing the fundamental concepts. 

The ideas and practices of what has come to be called 'mobile learning' date back to the turn of the 
century, to a small number of universities and organisations and to a handful of researchers, 
developers, activists and practitioners. Several years later the increasing power, functionality, 
coverage and ownership of mobile phones made them incredibly attractive to policy makers, 
programme managers, international agencies and humanitarian foundations working in the global 
South, wanting to provide educational opportunities and resources to people, communities and 
regions. 

There is a list of free and accessible resources in the end section, including those specifically relevant 
to the global South. 

The growth of mobile learning research, the growth of researchers working on mobile learning and 
the apparently attractive and benign possibilities of mobile learning suggest the need for much 
research to underpin any subsequent investment, and to underpin policy, capacity building and 
awareness raising. The current paper is intended to help the funders of research understand and 
optimize the research that they fund and the research findings that they use. 
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Mobile Learning Research 

This paper starts by giving policy makers and other people funding research a practical analysis of 
different types of mobile learning research in order to articulate more realistically the nature of the 
insights and findings that they should expect from the mobile learning research community. It also 
looks briefly at the changes and trends in mobile learning research in order to give a better sense of 
what to expect in future. 

The Nature of Research 

The word 'research' can be used to refer to finding information, for example from Google or 
Wikipedia, or to refer to market research, from people in the street. We, however, are talking about the 
activities of experienced professional researchers, typically based in a university or institute 
environment, who gather data in the field or in the laboratory. The purpose of their work is typically 
to test, endorse, refine or refute some theory or hypothesis. At the highest level of research, for 
example, for an academic journal or a research conference, we are looking at work that should be: 

• Original - it makes a unique and novel contribution, however small, to what we know, 
meaning some aspect of the research has not been done before 

• Applicable - it solves or answers an existing or anticipated problem or question, and is 
worthwhile 

• Generalisable or Transferable - it has some wider significance, it can be transferred into other 
contexts or locations 

• Repeatable - it is sufficiently documented that it can in theory be repeated and checked, 
ensuring its rigour, trustworthiness and soundness 

• Ethical - it 'does no harm' either in its outcomes and consequences or its methods. 

These claims or attributes must each be substantiated by a comprehensive review of the existing 
academic literature, and at this highest level, research findings should be guaranteed by various kinds 
of peer review, iterative revision, impact ratings and community scrutiny. 

These do perhaps represent a gold standard, most explicitly adopted in the universities of Western 
Europe. Standards differ and in many parts of the world, for instance the Middle East and the post- 
Soviet Republics, PhD research can largely focus on un-critical accounts of existing literature, and in 
East Asia respect for tradition or authority can hamper original thought. 

With an emergent discipline such as mobile learning, there will always be differences in expectations 
and interpretations from the diverse constituent disciplines, from education (or pedagogy or 
didactics), mobile technology (or computer engineering), development studies and others. In this 
respect, however, mobile learning research is maturing and the past decade has seen an international 
professional body emerge, the International Association for Mobile Learning, along with a recognized 
scholarly journal, the International Journal of Mobile and Blended Learning, and annual international 
research conferences, such as mLearn. There are also regional conferences, such as IADIS Mobile 
Learning in Europe or mLearnCon in US. There are fewer in the global South, perhaps only conferences 
where the commercial aspect competes with the scientific aspect, such as eLearning Africa. This in itself 
imposes a pervasive bias onto which research is presented, who presents it and who hears about it. 
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Processes and standards are in place to ensure and enhance the rigour and thus the trustworthiness of 
research findings. These research findings constitute the traffic from researchers to policy makers, 
their funders and any other stakeholders, including the wider public. They are however detailed, 
structured and highly contextualized, and by their nature, difficult for non-specialists to read with the 
necessary scrutiny, scepticism and criticality. 

This discussion of research may seem to have little significance to programme managers and policy 
makers. It does however articulate the standards, practices, drivers and culture that researchers 
inhabit, and helps us understand some of the causes of misunderstanding and miscommunication. 
Furthermore, research may have a number of roles or functions for policy-makers, donors and 
officials, and it is important to distinguish these and nurture appropriate expectations. 

The Types of Research 

There is a history of deploying new mobile technologies on a small scale for a fixed term to test 
technological or pedagogic concepts or to act as exemplars. The technologies are however often ahead 
of what the market, especially the disadvantaged and underprivileged parts of the market, can afford. 
These small-scale fixed-term projects may thus have only limited significance for policy makers and 
may present unacknowledged challenges when policy makers seek to transfer the procedures, 
technologies and formats from their original context. With these new technologies, there is a history of 
researchers undertaking research projects that exploit each new hardware, software or network 
technology as it comes to market, sometimes losing sight of the longer-term impact or conceptual 
significance of such projects. These projects can represent a distraction for policy makers as, not only 
might there be questions of sustainability, reliability and affordability, but they are also ahead of the 
curve in terms of wider acceptance and understanding. There can also be a problem of churn. This is 
the phenomenon of continually going back to the start and can distract from research that focuses on 
affordable sustainable innovation around existing robust and ubiquitous technologies; policy makers 
have a role and have a responsibility in directing researchers towards activities with the most socially 
useful outcomes. Research projects promoted as exemplars are problematic since they may exemplify 
ideas that are unaffordable or unsustainable. Research projects that seem to succeed but on the basis 
of devices supplied - lent or given - to learners are also likely to be promoting ideas that are 
unaffordable or unsustainable. The same is true for projects that depend on one-off donations, for 
example, message bundles from mobile network operators. These problems may not arise where the 
underlying assumption is of specific devices becoming cheaper or network coverage improving. 

There are occasionally research projects that propose to take a concept to scale. They are often but not 
always, preceded by smaller pilots. These larger projects should be designed to answer questions 
about wider community, organisational and institutional issues, not about pedagogy or technology, 
and should only proceed with robust, resilient and proven technologies and systems, adequate 
training and testing, and a clear understanding of the objectives and methods for gathering evaluation 
data. These are perhaps the most interesting and important research projects for policy makers, 
ministry officials and programme managers since they align most closely to their own work, interests 
and responsibilities. 

Large-scale deployments and evaluations do not however only measure reactions and test responses, 
they also have the capacity to create or destroy the expectations, attitudes and goodwill around 
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specific technologies and programmes and so must be planned and presented carefully. They should 
not be used to test untried technologies; these can be tested by simulation or in labs or with 
researchers, students or officials rather than the general public or vulnerable learners. 

These are only a few of the possible types of research. We could enumerate others, for example, blue- 
skies, developing theory, testing new pedagogy, implementing new technology, importing 
approaches or theories tried elsewhere and so on. With each of these types of research comes a 
particular kind of outcome, a particular kind of finding, and our point here for both researchers and 
policymakers is to be clear, sceptical and explicit about the expectations, relevance and nature of any 
particular research idea and approach. 

The Trends in Research 

Having looked at research in general, it makes sense to move onto the specifics of mobile learning 
research and how it is evolving, especially in relation to the global South. 

One undeniable observation is that the relatively upbeat global economic climate that supported the 
early mobile learning research community at the turn of the century is now downbeat, considerably 
reducing both the public sector and private sector support for research anywhere in the world. Over 
the same time, interest and understanding of mobile learning to support educational and 
humanitarian missions in the global South has increased, opening up the possibilities of research 
funding from international agencies, donors and foundations, and the mobile network operators have 
seen education as part of their business with a vast market in the global South. At same time we have 
seen mobile technology change from expensive, difficult, scarce and fragile to cheap, universal, 
robust, pervasive and ubiquitous, having the effect that whilst previously mobile learning needed 
research and researchers, now mobile learning has become easy, obvious, common-sense, apparently 
needing no further research. 

Technological research will always be the domain of technology corporations; they have the money 
and expertise to get ideas to market. What are constantly changing and always varied are the 
attitudes, expectations, abilities and experiences of learners as they engage with mobile technology, as 
it transforms the learning they need and the world around them. Whilst technology research, or rather 
technology development, is universal and not negotiable or culturally specific, research with learners 
is highly specific to their context and their circumstances and local to the responsibilities of regional 
and national policy makers and ministry officials. 

What is still not well understood is the larger context for technology and pedagogic change, 
specifically issues of scale and sustainability. Research is still needed but this research by its nature 
implies a greater commitment of resources than earlier short-term, small-scale research and requires 
continued scrutiny of context in the search for transferable, durable or generalised findings. This is 
partly because the issue of culture in mobile learning is so poorly researched and because most 
research has taken place against the un-remarked backdrop of English or Western culture, pedagogy 
and assumptions using global technologies and systems and so thus the issue of other cultural 
contexts seems to have seldom arisen. 
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Successful Research 

Sadly, the professional and financial environment in which researchers work can mean that failure or 
the appearance of failure is not an option and from the start of the research process, the bidding, to its 
conclusion, the reports and publications, failure seldom makes an appearance. Our experiences 
showed that the community was aware of the need to learn from mistakes and failures. Researchers 
do have the responsibility to publish their findings in order to extend our knowledge of mobile 
learning. However, this can conflict with perceptions that journal editors prefer positive results and so 
undocumented mistakes are perpetuated. 

The Ethics of Research 

We said earlier that a characteristic of research, or research that would be viewed as acceptable by 
universities, professional bodies, research journals, the press and the wider world, was that it should 
be ethical, that it should do no harm. 

For researchers from the universities of the global North, research on communities in the global South 
and mobile technologies can pose several sorts of ethical challenge, ones that address whether harm 
might be done. These include the adequacy of any institutional ethical processes, the novelty, 
complexity and abstractness of mobile technologies and the difference and distance of any target or 
host communities, especially poor, marginal or rural ones, from their own experiences, expectations 
and values. 

This connects with the Golden Rule vs. the Platinum Ride, the former telling us to treat others as we 
wish to be treated, superseded by the latter as people realized that how one might wish to be treated 
might not be the same as others would want to be treated. This difference can be connected to culture, 
religion, and societal values and norms —this was understood by many in the research community. 

The research community at the symposium revealed a number of ethical practices that should be 
considered while working with community groups or marginalized people, these include: 

• ensure participants understand their rights as subjects of research 

• ensure participants understand the power limitations associated with funders, especially 
corporate funders 

• follow the ethical clearance procedures from reputable institutions. 

• translate ethics documentation into local languages 

• empathize with participants respecting local culture, time and practices. 

These are only some operational hints. There are undoubtedly many more that are relevant. 

Organisational Practices 

Some organisations have explicit ethical clearance procedures (IRBs in USA); these may or may not be 
adequate for research with distant and different cultures using mobile technologies. Other 
organisations, sometimes ministries, corporations, NGOs, consortia, funders and consultants, do not 
have such procedures. Policy makers and others funding research should bear in mind that recipient 
organisations may have no ethics procedures or practices in place and consider their own role in 
creating appropriate expectations of the project lifecycle, from building in requirements for an ethical 
evaluation in proposals through to expecting to see ethics documented in publications and reports. 
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The majority of the participants in this study believed that ethical procedures should be different 
depending on the country; that target populations should be able to provide insights about their own 
ethics and that ethical procedures should have some flexibility and not merely represent the powerful. 

These procedures, even within the participants' own country, may be perceived as bureaucratic and 
paternalistic, producing only documentation and tokenism, and can be dominated by specific 
disciplines, for example, psychology or clinical medicine, and are thus unsuited to emerging areas of 
research of an interdisciplinary nature, ones where the nature of harm may be subtle and complex. It 
is possible to work within these procedures whilst also pressing for their improvement. 

Perhaps regrettably, organisational ethics procedures often prioritise a public relations dimension to 
their work and are keen to make decisions that protect and promote the image of their organisation. 
The well-documented media panics about education and mobiles, especially with younger children, 
may be in their minds. They can also be methodologically conservative or biased, and not open to 
research methods that are aligned to online research with mobiles or culturally sensitive research with 
indigenous or remote communities. 

Researching with Mobiles 

There has been a variety of initiatives and technologies that use mobiles in the conduct of research in 
order to capture data, and these could be used to explore learners and learning with mobiles. This 
would be an authentic alternative to more traditional face-to-face or paper-based methods. 

One research participant at the symposium described visionary educators as the catalyst for many 
mobile learning initiatives in schools. A concern raised at the symposium was how national guidelines 
were often developed from very little evidence. 

There are also projects to get greater insights into how people including learners use their mobiles. 
These are intended to give educators a greater understanding of how learning content and learning 
interactions can be designed for people on the move in a wide variety of situations. This research is 
highly contextualized and highly specific to individual groups and communities. 

The primary role of mobile learning researchers was a topic that provided the greatest variety of 
responses from the symposium participants. These included: 

• disseminate authoritative information to support content and best practice 

• test or apply theoretical frameworks 

• inform decision making 

• explore affordances for the benefits of learner 

• identify local contextual factors 

• conduct grass-roots needs analysis 

• establish models and frameworks 

• encourage wider debate 

• study the application of technology 

• demonstrate the unique value of mobile learning. 


Again, this is only an indicative list but it illustrates the diversity of positions and possibilities. 
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Researching with Different Communities 

Any society, community or culture is uniquely characterised by a specific balance between the formal, 
the established and the institutional on the one hand and the informal, the indigenous, the local, the 
vernacular on the other hand, and perhaps the counter-cultural, the subversive and the disruptive, 
too, amongst peer groups, communities, families, kinship groups and elders. This balance can 
correlate or explain differences in ethics across different communities, ethics in the shared sense of 
what is approved, acceptable, appropriate, allowable or permissible in terms of interaction, 
relationship, manners, exchange, humour, posture, language, discourse, fashion and behaviour in any 
community, in what constitutes harmful and what constitutes harmless. Individuals of course often 
belong to more than one community and probably aspire to be accepted by several others. Looking at 
the basis of the ethics in any community exposes the challenge confronting the procedures, practices 
and documents that attempt to regulate research interventions from a distant institutional culture and 
community. Countries and regions are also characterised by a multiplicity of religions existing side- 
by-side. These religions form the foundation of their communities' ideas about ethics and morality but 
differentiate each from the other. Furthermore, the rapidity of technological, specifically ICT, 
adoption, change and appropriation leads to generational differences, not always based on 
chronological age, overlaying other cultural or communal differences. 

Any of these norms and characteristics is very different from those of research institutions in the 
global North and may be unfamiliar to local officials and policy makers. From the perspective of 
outsiders, of researchers coming from outside, there is much potentially that might be not worth 
mentioning or be taken for granted and many areas where harm might be inadvertently perpetrated. 
This is with varying granularity and relevance to communities from nations and religions down to 
villages and families. Also researchers coming from outside may have an inappropriate perspective on 
ends and means. An example might help; a pilot in East Africa involved in issuing mobile phones to 
children in rural areas finished after six months. The researchers retrieved the mobile phones and the 
children were distraught, having become attached to the phones. The researchers' response was that 
they 'couldn't make omelettes without breaking eggs', in effect that the scientific ends justified the 
scientific means. 

Institutions with traditions and culture derived from the North and the West, because of their 
academic pre-eminence in the global South have been in a position to define research ethics globally 
and thus define harm on the basis of their own heritage and perspective. We should be aware of 
Western bias - confusing the responsibility to educate others with the right to educate them and we 
should see mobile learning research in this more balanced way. Perhaps when, in bygone days, 
delivering and supporting education was challenging and difficult then the moral imperative to 
educate was a useful spur. Now however pervasive and intrusive mobile technologies make it much 
easier to deliver and support education and the moral imperative to reflect first should come into 
play. Also, researchers from the global North can often assume that other cultures subscribe to the 
notion that informed consent need only come from the individual not from their community, or from 
elected representatives. This is a largely Western and individualistic notion. Community 
understanding of sophisticated and abstract mobile technology systems may grow slowly as research 
interventions progress. These are arguments for continued community involvement in ethics 
procedures, for continual communal consent rather than one-off individual consent. This is however 
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further complicated by talk amongst the development community of unexpected consequences - how is 
the potential harm of interventions to be judged when outcomes seem to be so unpredictable? 

Commercial Support for Research 

Commercial companies and corporations in the mobile learning space, for example handset 
manufacturers, mobile network operators, broadcasters and content publishers, have sponsored 
research, either directly, through their corporate social responsibility divisions or through their 
foundations. They may do this in cash or in kind, and the latter is likely to be their own products or 
services. They may insist on timescales that researchers find too short for meaningful results, and they 
may have no formal ethics procedures. 

Our research suggested that researchers are conscious of several issues. A concern was that the 
research conducted on mobile learning was connected with specific corporations providing that 
funding. 

Concerns were echoed in the symposium. 

Of course, in all the academic papers the record for research is behind the pay wall, it's difficult to 
reach out the evidence. 

A symposium participant from the business sector described how many vendors are looking for 
"proof of concept", in that they need evidence that will justify the investments they were making in 
the education sector. The symposium participant described the need to have new products for the 
market tested with pilot studies. 

Summing Up Research 

So in conclusion, most research formats have some relevance to policy makers, ministry officials and 
programme managers but the diversity of research formats that they might either fund or might 
review means that considerable judgment and discretion are needed when looking for the policy 
implications. 

Relevance of Data 

This section looks at data, specifically at the different types and sources of data that the mobile 
learning research community can produce, in order to assist policy makers in the judgements they 
must make about the impact and relevance of data to policy formulation and implementation. 

Research by its nature produces data. Data is however usually less trustworthy and relevant than 
policy makers and the public might like to think. There is a trade-off between relevance and 
trustworthiness. This is because trustworthiness will naturally increase if researchers minimize 
extraneous, confusing or confounding influences and factors. This however produces an increasingly 
artificial or simplistic research environment, like a laboratory, unlike most real situations. A specific 
example is the tendency in mobile learning research to issue learners with the same mobile device for 
the duration of the project. This was logistically straightforward, technically attractive and 
methodologically more rigorous since it reduced the confusion of getting outcomes from diverse 
devices. Unfortunately, the likelihood of sustainable mobile learning based on learners being given 
devices is remote and instead we see the BYOD, bring-your-own-device, policy as the way forward. 
Unfortunately, all our earlier evidence based on consistent supplied devices is not relevant to such 
situations. 


14 



Furthermore, constraints on time, effort and resources will always mean that there is less data than 
necessary. This means that researchers must always exercise judgment and skill in order to maximize 
interest, relevance and impact within these constraints. Sometimes this may mean trade-offs between 
the duration, spread, intensity and methods being deployed since research budgets are always finite. 
Various factors compete to consume resources so, for example, longer research must mean shallower 
research because intense research uses resources more quickly; likewise more sites cost more than 
fewer sites. So, whatever research formulation is chosen, it imposes limitations on the trustworthiness, 
relevance and generality of the findings. In particular, qualitative data will cost more to gather and 
more to analyse than quantitative data - quantitative data will tell you what people do but qualitative 
data will tell you why they do it. 

We should perhaps say something about qualitative data and quantitative data since these are 
fundamental to research methods and research findings. Put simply, quantitative is usually 
structured, numerical and objective whilst qualitative is textual and subjective. This means that 
quantitative data can reveal much about learners' behaviour but qualitative data is needed to explain 
it. Typical methods for gathering quantitative data include statistical methods, checklists, structured 
interviews, system logs and Likert scales whilst qualitative methods include semi-structured 
interviews, free-text surveys and focus groups but we should point out that these are conventionally 
face-to-face and not clearly aligned to mobile learners with mobile technologies. This means there are 
differences in how the data is obtained and thus in the cost of obtaining it. 

As expected, within the research community, there was no individual research method that was 
preferred for researching mobile learning. Good researchers let the research questions determine the 
methodology used. 

The research participants also describe a shift to include qualitative methods that can capture other 
less quantifiable evidence. A mixed methods approach was suggested by many of the participants - 
mixed methods describes research that collects both quantitative and qualitative data. 

I understand the taxpayer's wish or pressure again to see how money is spent. There should be a 
balance to be showed somewhere between, okay, results that can be quantified, also we have all 
these beautiful dimensions about, dimensions such as engagement, motivation, but also change, the 
cultural change in teaching practice, all the small elements that constitute what you say, the 
learning or teaching culture, the mind-set. This is something we cannot fully identify in quantitative 
thesis. 

The research suggests that quantitative methods may be the choice for political gain but do not form 
the research agenda that can serve the policymakers and practitioners. Policy makers might need 
research on: 

• the benefits of mobile learning 

• how technologies can support different forms of learning 

• the development of mobile learning curricula for all ages and subjects 

• scalability of existing versus potential solutions 

• appropriate use and policies 

• how to provide access to mobile learning for all learners 

• how mobile devices are for learning and not just a distraction. 

Many of these are policy-related rather curiosity-driven. 
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In assessing the relative costs and outcomes of different proposed ways of gathering research data (or 
evaluation data) in the field, policy makers and programme managers must recognize various trade¬ 
offs and reservations. 

We have ethical and methodological reservations about power differentials between the researchers 
and their subjects. It does not feel right and the data is not worth having. We have experienced 
evaluation exercises in East Africa where senior representatives of the Ministry of Education and of 
the overseas Department of International Development, in a convoy of very expensive vehicles, went 
into poor rural schools to ask primary school teachers what they thought of the programme in 
question. Clearly the differentials of power and authority made any responses and reactions 
worthless. 

In looking at the options for gathering research data in the field, there is a tendency to round up the 
usual suspects - that is, interviews, focus groups, questionnaires - and to avoid options that might be 
more imaginative and possibly more appropriate to mobiles, to people on the move, to people on the 
move learning informally and learning together, and to people who are not the usual categories of 
research subjects or correspondents. 

It is always sensible for funders to ask researchers not only about the choice of individual research 
techniques but to also ask about their sequence and their relationships. Was one technique used to 
refine or corroborate another one? Did system logs check users' recollections? Did a questionnaire or 
survey that gave objective data about behaviour lead to semi-structured interviews that provided data 
about reasons and motives? Did a broad survey identify areas for deep investigation? Did interviews 
identify topics for subsequent surveys? And so on. Policy makers reading researcher outputs should 
be asking questions about individual research techniques. Was it piloted? Is the documentation 
available? Did it use a recognized method? 

Settings for Data Collection 

This section also looks at data, specifically how the process of selecting, siting and sampling locations 
for mobile learning pilots and trials can constrain its value and impact in policy formulation, 
particularly at a regional and national level. 

Where trials, evaluations and pilots or samples and test sites get located usually heavily influences the 
findings, outcomes and conclusions, and certainly does not usually give the kinds of national or 
regional validity that policy makers and ministry officials might like it to, especially when resources, 
personnel and time are under pressure. It is tempting, for example, to think that locating a trial or 
collecting data from every province or district gives findings that are nationally relevant. Locating a 
trial or collecting data from each tribal or ethnic community or from each type of agricultural 
community, for example, pastoral, nomadic, subsistence arable and peri-urban market gardening, also 
seems to provide an equally convincing evidence base so we must ask what is going on when we site 
trials or sample different groups. What is going on is that we build in the assumption that the variable, 
for example, tribal group or administrative region, is important and must be accounted for, must be 
neutralized, by spreading the sampling across its range. This may be true but ignores other variables, 
perhaps age, income or gender as having a possibly larger impact on our findings. The message to 
policy makers is that findings that seem to have a national significance may be less trustworthy than 
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they seem at first and that extrapolation, moving from the data gathered in the field to some more 
general conclusions, may be unreliable however desirable and attractive it may be. 

How to choose a sample (group of participants) and a setting (the location) for data collection is an 
issue that all researchers have to face. The majority of the participants who chose to answer this 
question said that it was dependent on the research methods that have been chosen. For example, if 
qualitative methods are selected there may only be a few people in the study, perhaps two or three 
and this research may take place in a small village. If quantitative methods are selected, the numbers 
may be in thousands or more, however, this could take place in a city or with multiple villages 
depending on the criteria being used. The site chosen should be as similar as possible to the proposed 
generalized area; therefore, if the research is to better understand how high school students use 
mobile devices to learn about science, the study participants should be high school students. 

The Evolution of Mobile Learning as an Idea and a Subject of Research 

This section explores the various interpretations and definitions of the concept of 'mobile learning' in 
order to articulate the practical implications for mobile learning policy, and indeed mobile learning 
funding and implementation. Experience over the last decade has shown that this seemingly academic 
issue has considerable practical implications especially for the financing and regulation of mobile 
learning. UNESCO articulated a definition in its Policy Guidelines that could be used as a starting 
point. This section will place this definition in a wider context. 

There have been a variety of interpretations of what mobile learning might mean but here we attempt 
to draw the attention of policy makers, programme managers and officials to the implications of this 
variety. 

Mobile Learning as Technology 

The earliest definitions focused on mobile technology as the defining characteristic, for example 
saying that mobile learning "...was any learning where mobile or portable or handheld technology 
was the main or sole delivery mechanism." These definitions were problematic because technology, 
its ownership and its use always moved on, leaving earlier definitions looking out-of-date and 
underpowered. There can also be problems with policy-making that rests on techno-centric 
definitions, e.g., as the basis for funding priorities or regulatory frameworks. One example is the UK 
MoLeNET programme for the TVET sector, amounting to some 14m GBP over three years and 
devoted to hardware and infrastructure. This defined mobile learning in terms of mobile technologies 
and then encountered problems adjudicating on exactly which maximum screen size constituted a 
mobile device, and also fighting the then-fashion for the UMPC, the so-called ultra-mobile PC of 2006. 

Mobile Learning as Learners Moving 

Subsequent perceptions and definitions have shifted on to focus on the mobility of the learning and 
the learners, especially when they move across contexts, settings and situations. The focus has shifted 
to initiatives within education systems and their institutions to enrich and extend the existing ideas 
and practices of learning. The former has often sought to deploy high-end and emergent technologies 
within small research contexts using technology supplied to the learners. This has produced 
interesting and rigorous results but ones that are only trustworthy in situations where learners can be 
supplied with the necessary technology. The latter has used mobile technology to reach people and 
communities for whom other educational interventions would be too costly, difficult or dangerous. 
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This obviously includes rural areas, sparsely populated areas, areas of poor infrastructure and 
challenging environment, and areas with no secure buildings and human resources to support any 
other technology or delivery. It also includes reaching marginal, sometimes stigmatized, communities, 
people from communities without the money, confidence or familiarity to exploit any other 
technology or delivery of education, and people unable to access other interventions because of 
cognitive or physiological disadvantages or disabilities. This perspective has one set of policy 
implications, and continues to be the dominant paradigm. It works with the existing institutions, 
professions, curriculum and ministries. It is however problematic in the context of fixed, stable, static 
institutions and professions in mobile, moving societies. 

Mobile Learning in Mobile Societies 

There is however also a cluster of definitions that emphasis the centrality of mobility and 
connectedness as a defining characteristic of modern societies and cultures anywhere; these 
definitions place learning with mobiles outside formal education; they see learning with mobiles as 
only a small part of people's mobile lives and their lives with mobiles; they focus on the capacity of 
people and their mobiles to generate, share, transform, discuss ideas, information and images, that is, 
the capacity of people to shape their own mobile learning. 

These different perceptions and definitions have very different implications for research methods, and 
for policy priorities; and for relationships between policy-makers, researchers and types of learners, 
their motivations, goals, attitudes and styles. 

Many ministries and institutions are addressing the sustainability of mobile learning by considering 
BYOD (bring-your-own-devices) policies for learners, thereby displacing the expense of handsets and 
connectivity onto learners themselves but introducing learner agency, control and expectations into 
the equation. 

Given that most schools in most countries have out-right prohibitions of mobile devices still in place, 
the current policy environment is something of a muddle. Research into attitudes, expectations and 
pedagogies should all be on the policy makers' agenda if they want effective policy and sustainable 
deployment and implementation of learning with mobiles. 

Some schools, usually acting on their own initiative rather than responding to national or regional 
prompting, have engaged in discussions with their learners designed to devise a new consensus about 
bringing mobile phones to and using them in schools, recognising that older children may be the head 
of a family deprived of parents and perhaps recognising the futility of on-going prohibition. 
Increasingly, there is software in use capable of enforcing a more nuanced policy for mobiles on 
school premises. 

There is also vast amount of what we might call user-generated learning, that is, content, 
conversations and communities produced by learners or users themselves rather than by institutions, 
agencies and ministries. This represents a vast resource and opportunity for policy makers to exploit 
once they have used research to scope and map it, and to develop processes to monitor, endorse and 
publicise it. It consists, for example, of podcasts, Facebook groups, smartphone apps, web sites and 
social networks but is constantly changing and sometimes of dubious quality and doubtful 
provenance. 
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Researching Mobile Learning 

This section looks at the challenges and pitfalls inherent in different kinds of mobile learning research 
and how these different kinds are evolving, in order to calibrate expectations for policy makers. The 
nature of the ethical dimension of mobile learning and mobile learning research are explored in order 
to help policy makers develop more robust, consensual, appropriate and transparent mechanisms 
designed to reduce the risk of harm, especially in programmes and interventions with the 
disadvantaged and the marginal. 

The following sections address briefly a succession of questions, 

• How is education researched? 

• How is mobility researched? 

• How is mobile learning researched? 

• Why is mobile learning research different and difficult? 

Researching Education 

Educational research is difficult. This is because education is complex, situated and life long, making it 
difficult to attribute educational (or personal, social, economic or political) success in later life to 
specific educational interventions. These successes are in fact always multi-causal and they are often, 
to use a phrase popular in international development, 'unexpected consequences'. Caution amongst 
educational policy-makers has perhaps led to preference for overly 'scientific' or empirical methods, 
ones that treated education as a laboratory experiment, and for a predominance of short-term pre¬ 
test/post-test perspectives. Policy makers need to be aware of the complexity of the phenomena that 
they commission researchers to explore. 

There is furthermore a global outbreak of over-testing and this sadly might seem to offer masses of 
free data for researchers to explore on behalf of policy makers. 

In general, there is always a risk of measuring what is easy, cheap or quick to measure and avoiding 
measuring what is difficult, expensive or sustained. This can mean that simple, numerical and 
quantitative data and surveys are preferred to complex, qualitative and textual data. Technological 
systems and online methods, for example mobile phone network infrastructure and users' handsets, 
can produce masses of 'cheap' and apparently independent data whilst semi-structured interviews are 
skilled and expensive. This can lead to a focus on objective outcomes and behaviours at the expense 
any understanding of the motivations and values that lay beneath them, and can lead to attributing 
effects to the causes that were measured rather than the ones that were not. 

It is easier to measure formal learning than informal learning since the inputs, objectives, outcomes 
and environment can be documented and controlled. It is easier to measure the impact of training 
than of education, easier to measure skills acquired and knowledge acquired than it is to measure 
confidence gained and attitudes changed. It is easier to measure lower order skills than higher order 
ones. These remarks take on increased significance when we think of the potential of mobiles for 
informal learning, adult learning and community learning, and when we think of the potential in 
accessing the wealth of online resources, communities and ideas through the portal of the mobile. 
Employers globally talk of the economic importance of increasing entrepreneurship and building 
'soft' skills, both areas where research and measurement are more difficult than similar work on 


19 



traditional 'hard' skills, and experts globally talk of the need for lifelong learning. Mobiles are a 
universal, sustainable and effective technology for supporting, enhancing and delivering them all but 
policy makers need specific research to identify the best interventions, priorities and resources to 
deploy. 

Researching Mobility 

Mobility research, meaning research on the social and economic impact of mobile technologies and 
the mere fact of moving, is difficult. Mobility is a commonplace part of everyday life, not unusual, 
separate and isolated like using computers. Outcomes, changes and impacts are yet again multi- 
causal, long term and diffuse; they are social and affective, not just cognitive and objective; whilst 
researchers talk of unexpected consequences, people do not talk about undocumented causes. 
Credible, authentic and naturalistic research is difficult - researchers are still struggling to observe 
and measure social interactions and social practices without inhibiting and distorting what they are 
trying to observe. More powerful and more compact and portable technologies make this increasingly 
easier but never as easy as research on stationary people in static situations. In general, researchers 
know very little about contemporary mobile digital habits, especially in marginal or remote 
communities. The rapidity of technical change, the rapidity with which different cultures adapt and 
adopt the technology and the diversity and fluidity of these different cultures means that educators 
and policy makers have little idea about the ways in which mobiles fit into the lives of people, 
especially the lives of people least like themselves. 

One participant at the symposium echoed this sentiment, saying, 

I think it is also important to note that the research agenda for mobile learning is going to be much 
different, the devices are different, the scale is much larger, but also the contextual use is different. 

So whereas I may have approached E learning as part of a formal course, there can be social and 
informal learning which is one of the hallmarks of mobile learning. 

Fortunately, there is a growing body of research, under the banner of m4d, mobiles-for-development, 
looking at the social and economic impact of mobiles on people in the developing regions of the 
world, the impact on fisher-folk, subsistence farmers, small traders and the sick. There is for mobile 
learning researchers and policy makers the need and the challenge to reach across to these other 
research communities; working with farmers, fishers or the sick represents an opportunity to develop 
and deliver mobile learning that is meaningful and integrated within the needs and aspirations of 
specific communities. Policy makers need to be aware of the breadth of inter-related disciplines from 
which they could commission research. 

There is also a wealth of technical data available from networks and from devices that might for 
example give all of the SMS traffic through a dedicated support system or details of context, that is, 
time and location, of selected user activity. This kind of research may be cost-effective in terms of 
coverage and scale but is highly technical, in terms of accessing the data and storing it and analysing 
it, and ethically probably problematic, because of perceptions of surveillance and risks to 
confidentiality and anonymity but nevertheless has lots of potential. 

Mobile technology is pervasive, ubiquitous, nearly universal and potentially intrusive. As the vehicle 
for educational interventions, it has infinitely greater capacity to reach into people's lives than 
computers or televisions and to reach beyond the traditional systems, institutions and buildings of 
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education and into families and communities; consequently the ethical dimension of mobile learning 
research and mobile learning policy must be an integral component of research, policy and practice. 

Research Funding and Incentives 

This section gives policy makers an overview of the context and background in which the mobile 
learning research community operates in order to understand how these contextual factors shape the 
capacity of the research community to contribute to policy. We look briefly at how research is 
managed, funded, reviewed, staffed and resourced, how researchers' careers, opportunities and 
incentives are structured. These issues have a bearing on what research gets done or not done, and 
how it gets done, but also on how policy makers should view their relationships and interactions with 
researchers. 

Researchers 

Most researchers work in universities or institutes, though whether these are in the global North or 
the global South makes a vast difference. Researchers are always under pressure to bring in extra 
funds and will bid for external projects and programmes. To build a research career they must be 
successful in bidding and in the global North their institute or university will have a sophisticated 
infrastructure to help them win bids. Compared to researchers in the global South, however, those in 
the North are much more highly paid and have travel and subsistence costs that are higher than their 
competitors in the universities and institutes of the global South. We see here the outlines of a cycle 
whereby successful researchers, most likely in the global North, become more successful, because of 
their track records, and other researchers, more likely in the global South, have trouble breaking into 
this cycle. The Strengthening Information Society Research Capacity Alliance (SIRCA) programme of 
IDRC, the International Development Research Centre, is one example trying to build capacity in the 
global South to break into this cycle. 

For a research career in the global North, publications in respected journals are recognized steps in 
professional development and advancement and are part of developing a professional portfolio. 
Publication in research journals is sometimes rewarded in developing countries but usually not, 
especially not for novel and emergent disciplines. There must also be an open question about whether 
peer review (usually of the researchers in the global North by the researchers in the global North) 
reinforces certain perspectives and positions and ignores others. Many developing countries have 
capacity problems in relation to academic writing, research supervision, proposal writing and in 
managing projects and consortia so that the development of any research domain and research culture 
is skewed in favour of the voices, the players and the opinions of the global North and consequently 
the impact on policy making is skewed too. Conference attendance and conference presentation are 
often seen as stepping-stones to publication, as well as networking opportunities for subsequent 
writing, consortia and bidding. Lack of expertise and experience in hosting and managing research 
conferences means that relatively few take place in developing countries. Furthermore, these countries 
have limited budgets for academic travel so attendance at overseas research conferences is limited. 
These are part of the wider concerns for research in the global South. 

Building Local Researcher Capacity 

The academic career structure in many developing countries does not incentivize, promote or reward 
research (as opposed to teaching or management). This means that local research capacity and 
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expertise develop very slowly, perhaps producing a reliance or dependency on external and foreign 
researchers (or less qualified consultants acting as researchers). This might be acceptable for 
technology projects but perhaps not for social science projects, those that involve engaging with local 
users, officials, managers and other stakeholders. Many foreign or external researchers need to use or 
choose to use local research assistants or local institutional partners. This is one way in which local 
capacity can be built. It does however raise questions about whether foreign or external researchers 
should be required as a condition of their funding to work with local people and local partners, and 
additional questions about whether they should earmark part of their time and budget to training and 
mentoring activities. Other ways in which local capacity can be built are research exchanges, bursaries 
and scholarships at universities and institutes in the global North. These are valuable but only if there 
are mechanisms to embed the experience and expertise afterwards and only if there is a critical and 
on-going exploration about which knowledge, values, perspectives and techniques from the global 
North are transferable and relevant and which are not. 

Many experts believe that bringing in local researchers is important. The research community had 
plenty to describe the positives gained from having local researchers involved in a study, saying: 

It is absolutely important to employ/involve local researchers in addition to external researchers 
because local researchers will be able to bring local socio-cultural contexts in research data 
collection and analysis process which will be valuable to produce credible research evidence on 
what works and what doesn't work. Moreover, local researchers can improve their skills and build 
their confidence in conducting similar research in the future, they can play a critical role in 
influencing policies by reaching out to decision makers and sharing research findings. 

Other benefits include the local researcher having the ability to: 

• provide in-depth knowledge of the learning ecosystem 

• address, recognize and understand local sociocultural issues 

• develop locally relevant research practices and activities 

• provide insight into commercial competitiveness 

• provide perspectives of the consumers of technologies and educators 

• provide insight into security or conflict issues in the area. 

These are issues that policy-makers can help with throughout the research project life-cycle. 

Summing Up Researchers 

For policy makers working in developing countries, these very general remarks have a range of 
implications. The first is merely the need for heightened awareness of the factors at work when they 
deal with researchers and research, the second may be a call to review how research is funded and 
how research calls are constructed, selected, monitored and integrated into longer-term strategic 
planning and policy making. These implications do not probably have any extra relevance to learning 
with mobiles in developing countries except insofar as this is relatively suddenly a popular topic with 
perhaps less cumulative critical experience and expertise, and with considerable popular, political and 
media interest. 

Concluding Remarks 

This has been a largely personal and informal attempt to make sense of the relations and interactions 
between researchers and policy-makers, both of them working with mobile technologies in the global 
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South to support and deliver learning. The two communities both seek in their different ways and 
with differing emphasis to increase what we know and to use what we know for the benefit of the 
disadvantaged and the marginal so any attempt to improve this work must be worthwhile and a 
valuable foundation to more detailed follow up. There is clearly work that can be done at various 
levels. At the highest level, this work should build on the symposium described here, bringing 
together more researchers and more policy-makers and exploring the observations and impressions 
that we document here, perhaps as a community of practice, under the aegis of the Commonwealth of 
Learning or UNESCO, or national agencies (and funders) such as DFID, USAID or IDRC. This would 
trickle down to lower levels and specialist communities. 
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